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Abstract 


This in-brief technical report documents the results from two different analytic approaches for 
examining the reliability of the slope for easyCBM* math measures in Grades K-8. Results 
varied by grade, assessment measure, and the analytic approach. Results patterns are discussed. 
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In-Brief: Reliability of the Slope of the easyCBM® Math Measures 
Background 


The National Center on Intensive Interventions (NCI; https://intensiveintervention.org) 
evaluates both screening and progress monitoring tools for their technical adequacy, with the 
goal of helping educators select appropriate tools to meet their needs. Many of the analyses 
required by NCII are already part of our standard practice in developing assessments and are thus 
described in detail in the many technical reports we publish as part of our assessment 
development process. Other analyses, such as the reliability of the slope, reported here, fall 
outside the scope of our standard technical reports. In this brief, we present the results of our 
analyses of the reliability of the slope conducted with a population of students in need of 
intensive intervention. 


Methods 


Sample 

The analytic sample consisted of students who took easyCBM" math progress monitoring 
measures during the 2014-2015, 2015-2016, and 2016-2017 school years. All students in the 
sample were identified by their districts as needing intensive intervention in the specific skill 
area targeted by the assessments for which their data were included in this study. Data from this 
study are a subset of a much larger extant data set. The larger data set includes scores for all 
students in all districts with easyCBM" accounts covering fall of 2014 to spring of 2017. From 
this larger data set, we included only those students identified as needing intensive intervention 
who had a minimum of 10 assessment scores for a given assessment measure with a minimum of 
20 weeks between the first and last administration occasion. Thus, sample sizes varied greatly by 
grade and assessment measure administered. 


Analyses 
We analyzed the reliability of the slope using two approaches, Pearson split-test correlation 
analysis and reliability of the slope. 


Pearson split-test correlation analysis. For each student, assessments were divided into 
two data subsets comprised of odd and even numbered tests, respectively, depending on the 
chronological order in which they were taken. An OLS slope of improvement (growth) was 
estimated for each data subset and for each student. 


Reliability of slope. Reliability of the slope is defined here as the ratio of the true score 
variance to the total variance. The true score variance is the random slope variance in a mixed- 
effects growth model (Ime4 package; Bates, Maechler, Bolker, & Walker, 2015) in the R 
software environment (R Core Team, 2018). The total variance is the estimation of total variance 
of each student’s individual slope of improvement (R Core Team, 2018). 
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Results 


Results for both analytic approaches (Pearson split-test correlation and reliability of the slope) 
are presented for Grades K-8 in Tables 1-9, respectively. 


Table 1 
Grade K Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Measure aun coefficient 
BRrOne (r) Lower Upper 

9 PC -.30 -.81 45 
CCSS Math 

9 RS 28 00 1.00 
Numbers and 31 PC .62 133 80 
Operations (NumOp) 31 RS 51 ey) 97 

PC NA NA NA 

Geometry (GEO) 

1 RS NA NA NA 
Measurement NA re NA NA NA 
(MEAS) NA RS NA NA NA 


Note. NA = Not analyzed. “Lower bound of the confidence interval around the median 
correlation estimate falls below 0.50 but meets or exceeds 0.40. 
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Table 2 
Grade I Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 
Analytic Correlation 95% Confidence Interval 
Measure ny h coefficient 
Peep oe (r) Lower Upper 

4] PG -.05 -.35 .26 
CCSS Math 

4] RS .10 .00 44 
Niumbeteand AGE Pe slit -.12 a3 
Cos (INTEL'S) 77 RS 18 00 AS 

22 PC 42 -.01 ral 
Geometry (GEO) 

2D RS Ws O01 .63 
Numbers Operations 20 PC ag -.08 70 
and Algebra 


Note. Lower bound of the confidence interval around the median correlation estimate falls 


below 0.50 but meets or exceeds 0.40. 
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Table 3 
Grade 2 Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Measure eure! coefficient 
Perens (r) Lower Upper 

51 PC .34 07 56 
CCSS Math 

9 | RS 50 22 .88 
Nuvibesand 45 PC oo 1 .62 
oon NaC) 45 RS 57 28 97 
Measurerient 7 PC 48 -.43 al 
Ee, 7 RS 01 00 70 
Numbers Operations 9 PC 03 265 68 
and Algebra 
(NumOpAlg) 9 RS .06 .00 Ge 


Note. Lower bound of the confidence interval around the median correlation estimate falls 
below 0.50 but meets or exceeds 0.40. 
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Table 4 
Grade 3 Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Measure pene! coefficient 
PEP aS (r) Lower Upper 

22 PC at9 -.26 56 
CCSS Math 

22 RS 23 .00 ‘95 
Nimber ond is PC 20) -.35 .65 
Operations (NumOp) 16 RS 09 NA NA 

5) PC -.08 -.90 .86 
Geometry (GEO) 

5 RS 36 .00 1.00 
Numbers Operations 15 PC 13 - Al .60 
and Algebra 


Note. NA = Not analyzed. “Lower bound of the confidence interval around the median 
correlation estimate falls below 0.50 but meets or exceeds 0.40. 
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Table 5 
Grade 4 Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Measure Analytic coefficient 
Approach (r) Lower Upper 

24 PC 26 -.16 .60 
CCSS Math 

24 RS 24 01 77 
Numbers and 31 PC als -.46 24 
Operations (NumOp) 31 RS 04 00 33 
Measurement NA PC NA NA NA 
(MEAS) NA RS NA NA NA 
Numbers Operations 16 PC 0) Sey 64 
and Algebra 


Note. NA = Not analyzed. “Lower bound of the confidence interval around the median 
correlation estimate falls below 0.50 but meets or exceeds 0.40. 
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Table 6 
Grade 5 Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Measure puerta! coefficient 
PENee (r) Lower Upper 
19 PC 31 -.17 67 
CCSS Math 
19 RS 42 00 1.00 
69 PC 23 -.01 44 
Numbers and 
O ti O 
AG OL) 69 RS 29 07 58 
Geometry 6 PC 44 -.58 92 
Measurement and 
Algebra 
(GeoMeasAlg) 6 RS 84 23 1.00 
Numbers Operations 6 PC .28 -.69 .89 
and Algebra 
(NumOpAlg) 6 RS 46 .00 1.00 


Note. Lower bound of the confidence interval around the median correlation estimate falls 
below 0.50 but meets or exceeds 0.40. 
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Table 7 
Grade 6 Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Measure n eure! coefficient 
PENS (r) Lower Upper 
5 PC 65 -.55 97 
CCSS Math 
5 RS .76 01 1.00 
Niumbeteand 8 Pe Al -.42 87 
Operations (NumOp) 8 RS 18 00 1.00 
NA PC NA NA NA 
Algebra (ALG) 
NA RS NA NA NA 
Numbers Operations 4 PC 50 -.89 99 
and Ratios 


Note. NA = Not analyzed. ‘Lower bound of the confidence interval around the median 
correlation estimate falls below 0.50 but meets or exceeds 0.40. 
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Table 8 
Grade 7 Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Measure puerta! coefficient 
Perens (r) Lower Upper 

3 PC .88 NA NA 
CCSS Math 

3 RS 62 NA NA 
Numbers Operations ») PC NA NA NA 
Algebra and 
Geometry x : 
(NumOpAlgGeo) 2 RS NA NA NA 
Measurement NA PC NA NA NA 
Geometry and 
Algebra 
(MeasGeoAlg) NA RS NA NA NA 
Numbers Operations 12 PC 58 01 .87 
and Algebra 
(NumOpAlg) 12 RS 29 .03 .90 


Note. NA = Not analyzed. * Lower bound of the confidence interval around the median 
correlation estimate falls below 0.50 but meets or exceeds 0.40. 
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Table 9 
Grade 8 Pearson Split-test Correlation (PC) and Reliability of Slope (RS) Analyses Results 


Correlation 95% Confidence Interval 


Analytic 


Measure coefficient 
Approach (r) Lower Upper 

86 PC Al 21 57 
CCSS Math 

86 RS .65** A3 92 

24 PC DY -.15 61 
Algebra (ALG) 

24 RS 38 05 93 
Geometry and NA PC NA NA NA 
Measurement 
(GeoMeas) NA RS NA NA NA 
SEEMS 32 PC 45 1B 69 
Numbers Operations 
and Algebra 
(DANumOpAlg) 32 RS 44 12 88 


Note. NA = Not analyzed. ‘Lower bound of the confidence interval around the median 
correlation estimate falls below 0.50 but meets or exceeds 0.40. 


Discussion 


Results varied by grade, measure, and analytic approach. We document reasonable reliability of 
the slope, whereby the lower bound of the 95% confidence interval around the median 
correlation estimate falls below 0.50 but meets or exceeds 0.40, for the Grade 8 CCSS Math 
measure using the reliability of slope analytic approach. Results for all grades and measures 
varied greatly and were, on average, relatively less encouraging. Given extant data were used for 
the analyses reported here, we are currently planning a series of studies to better control for 
sample population characteristics as well as instructional/assessment approach. For many grades 
and measures, for example, correlation coefficients were not estimable or reliable given 
extremely low sample size. We anticipate improved reliability of the slope results for all grades 
and measures in planned studies. 
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